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INTERACTIVE  MULTIMEDIA  COURSEWARE 
DEVELOPED  BY  INSTRUCTORS 


BACKGROUND 

This  report  describes  the  results  from  classroom  evaluations  of  interactive  multimedia 
courseware.  Ten  instructors  from  community  colleges  across  the  state  of  Texas  were  involved  in 
the  project.  Each  instructor  was  required  to  (a)  attend  two  workshops  to  learn  to  develop 
interactive  multimedia  courseware  (ICW)  using  the  Experimental  Advanced  Instructional  Design 
Advisor  (XAIDA),  (b)  develop  at  least  one  lesson  using  XAIDA,  (c)  use  the  courseware  in  their 
classroom  for  purposes  of  evaluating  its  instructional  effectiveness,  and  (d)  train  a  colleague  to 
use  XAIDA. 

Several  instructors  developed  more  than  one  lesson  during  XAIDA  training.  The  lessons 
used  in  the  evaluation,  the.  developers,  and  the  campuses  involved  in  the  project  are 

listed  below  in  Table  1 . 


Table  1.  Lessons  used  in  the  Classroom  Evaluation,  the  "  Developers,  and  the  Campuses 


Lesson  Title 

•/Department 

Campus  /Location 

Confined  Space  Entry 

Occupational  Safety  and  Health 

Odessa  College,  Odessa 

Bank  Reconciliation 

Marketing  and  Management 

South  Plains  College,  Lubbock 

Administration  of  Oral  Medication 

Nursing 

Southwest  Texas  Junior 
College,Uvalde 

Simpson  Analog  Meter 

Multi-modal  Transportation 

Southwest  Campus, 

St  Philip’s  College 

San  Antonio 

Creating  Webpage  Links 

Technical  Communications 

Tomball  College,  Tomball 

Linear  Equation: 
Slope-intercept  Form 

Math 

Navarro  College,  Corsicana 

Administering  a  Z-track  Injection 

Nursing 

Mission  del  Paso  Campus, 

El  Paso  Community  College 

El  Paso 

How  to  Install  RAM 

Computer  Information  Systems 

Amarillo  College,  Amarillo 

Taking  Blood  Pressure 

Instructional  Technology 

St.  Philip’s  College, 

San  Antonio 

Catalog  and  Directory  Structures 

Computer  Information  Systems 

Brookhaven  College, 

Dallas 
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The  Experimental  Advanced  Instructional  Design  Advisor  (XAIDA) 

XAIDA  is  a  system,  developed  under  the  sponsorship  of  the  Air  Force  Armstrong 
Laboratory  to  explore  techniques  for  automatically  generating  ICW.  XAIDA  consists  of  a 
program  called  Develop  for  entering  and  editing  lesson  topic  descriptions  and  a  program  called 
Deliver  for  presenting  instruction  on  the  topic.  Develop  provides  a  subject-matter  expert,  in  this 
case  a  community  college  instructor,  with  facilities  for  describing  the  structure  of  a  topic, 
describing  its  characteristics  and  behavior,  associating  multimedia  with  the  lesson  content,  and 
configuring  practice  exercises.  Deliver  provides  students  with  an  overview  of  the  subject  matter, 
a  detailed  presentation  of  lesson  material,  selective  review,  and  automatically  generated  practice 
exercises. 

Instructional  Materials 

Materials  used  to  communicate  with  students  are  required  as  input  to  XAIDA,  in  addition 
to  the  structure  of  the  topic  knowledge.  These  materials  consist  mainly  of  data  associated  with 
each  part  or  section  in  the  lesson.  For  instance,  each  section  requires  a  textual  description  and 
visual  representation.  Multimedia  of  all  types  can  be  used  as  instructional  materials  at  various 
points  in  the  lesson. 

Instructional  Procedures 

Students  use  Deliver  to  access  the  ICW  generated  by  XAIDA.  The  ICW  consists  of  a 
lesson  introduction  and  body.  The  introduction  is  a  sequence  of  textual  information  and 
multimedia  that  the  developer  assembled  to  gain  the  students’  attention,  activate  prerequisite 
knowledge,  and  motivate  them  to  learn  about  the  lesson  topic.  The  first  instructional  elements 
presented  in  the  introduction  is  a  graphic  introducing  the  lesson  followed  by  a  description  of  the 
lesson’s  objective.  The  last  element  in  the  sequence  is  an  outline  that  shows  the  structure  of  the 
topic  being  taught.  The  lesson  body  proceeds  sequentially  through  the  outline.  Once  an  item  has 
been  presented  to  the  student  the  topic  is  available  for  review. 

Practice  exercises  are  normally  invoked  whenever  a  student  completes  a  lesson  or 
subsection  within  a  lesson.  Practice  consists  of  interactive  exercises  that  address  all  of  the  facts 
contained  in  the  lesson.  Eleven  different  types  of  exercises  are  available  which  provide  practice 
on  recall  and  recognition  of  facts.  Examples  of  exercise  types  include  true-false,  multiple-choice, 
and  fill-in-the-blank.  Practice  is  fully  supported— students  can  answer  on  their  own,  return  to  the 
lesson  to  look  up  an  answer,  ask  the  system  to  provide  an  answer,  or  skip  the  exercise. 

Deliver  dynamically  constructs  each  exercise  from  the  knowledge  database  input  by  the 
developer.  Developers  need  only  specify  the  facts  that  the  student  is  to  master.  Exercises  are 
generated  under  the  control  of  a  miniature  Intelligent  Tutoring  System  (ITS).  The  ITS  maintains 
a  model  of  the  student’s  knowledge  that  tracks  facts  that  appear  to  have  been  mastered  and  any 
misconceptions  that  have  been  exhibited.  The  tutor  generates  exercises  that  tend  to  address 
unknown  facts  and  misconceptions. 

This  report  is  a  comprehensive  examination  of  the  instructional  effectiveness  of  XAIDA 
courseware  developed  by  community  college  instructors  from  eight  different  educational 
domains.  The  domains  include  occupational  safety  and  health  technology,  marketing  and 
management,  nursing,  multi-modal  transportation,  math,  computer  information  systems,  and 
instructional  technology. 
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METHOD 


Instructional  effectiveness  of  the  XAIDA  courseware  was  evaluated  on  three  fronts — how 
much  the  students  learned,  the  efficiency  with  which  they  learned,  and  how  they  reacted  to  the 
learning  experience. 

Minimum  computer  requirements  for  running  XAIDA  Deliver  were  met  at  all  college 
campuses.  All  computers  used  in  the  evaluations  were  at  least  486  66  MHz  with  Windows  95 
operating  systems,  with  the  exception  of  Amarillo  College.  The  computers  at  Amarillo  College 
ran  under  Windows  NT.  XAIDA  has  not  been  systematically  tested  running  under  Windows  NT. 
The  majority  of  computer  labs  ran  an  application  that  protected  the  hard  disks  as  “read  only.” 
These  applications  had  to  be  disabled  in  order  to  run  the  Deliver  program. 

Aspects  of  instruction  that  influence  learners  and  courseware’s  learning  potential  were 
investigated  across  the  classroom  evaluations.  The  results  are  organized  around  these 
instructional  aspects.  They  included:  (a)  contextualized  instruction,  (b)  impact  of  ICW  on  task 
performance,  (c)  computer-assisted  instruction,  (d)  training  staff  and  faculty,  (e)  use  of  humor  in 
instruction,  and  (f)  hands-on  practice. 

Evaluation  Instruments 

Evaluation  booklets  were  constructed  to  assess  the  instructional  effectiveness  of  the 
courseware.  Booklets  contained  evaluation  instruments  constructed  and  customized  for  each 
classroom.  Refinements  to  the  instruments  were  made  across  evaluations;  therefore,  not  all 
students  responded  to  identical  items. 

The  final  version  of  the  evaluation  booklet  contained:  (a)  pre-measures  of  comfort  using 
computers  to  learn,  self-rated  knowledge  of  the  lesson  topic  and  confidence  applying  that 
knowledge,  and  instructional  preference  [self-paced  computer-based  training  (CBT),  group-paced 
instructor-lead  class  lecture,  or  instructor-paced  computer-assisted  learning  (CAI)];  (b)  pretest  of 
factual  and  applied  knowledge;  (c)  written  instructions  on  how  to  open  a  lesson,  sign-in,  and  tips 
on  navigating  the  XAIDA  interfaces;  (d)  posttest;  (e)  post-measures  of  computer  comfort, 
knowledge  and  confidence  in  the  lesson  topic  area;  and  (f)  a  training  assessment  survey  (see 
Appendix  A. 

The  training  assessment  survey  was  designed  to  measure  students’  reactions  to  their 
learning  experience.  The  survey  contained  12  items  that  represent  aspects  of  sound  instruction 
and  the  post-measure  of  instructional  preference.  Students  rated  the  12  items  based  on  their 
learning  experience.  All  ratings  were  made  on  a  7-point  scale  anchored  by  reciprocal  descriptors 
(e.g.,  sufficient/insufficient,  adequate/inadequate,  added  to/distracted  from).  The  final  item  on  the 
survey  was  a  post-measure  of  instructional  preference. 

The  pretest  and  posttest  were  equivalent  forms  and  counterbalanced  across  booklets.  The 
booklets  were  randomly  distributed  to  students. 

Measures  of  courseware  efficiency  were  collected  from  two  sources,  student  data  and 
automatically  from  XAIDA.  Presumably,  each  courseware  module  was  equivalent  in  content  to  a 
50-minute  classroom  lecture.  Students  participating  early  in  the  evaluation  project  entered  their 
lesson  start  and  stop  times  in  the  evaluation  booklets.  Time  on  task  was  computed  from  the  data 
provided  by  the  students.  Later  in  the  evaluation  project,  time  on  task  was  computed  from  journal 
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files  generated  by  XAIDA  Deliver.  Journal  files  are  automatically  created  when  users  interact 
with  XAIDA.  A  file  contains  a  time  stamped  record  of  interactions  between  the  user  and  the 
interface.  Thus,  a  journal  file  contains  entries  indicating  when  a  lesson  is  opened  and  closed. 

RESULTS 

Results  from  eight  classroom  evaluations  are  reported  in  this  section.  The  courseware  and 
approach  taken  in  each  evaluation  are  described  below  along  with  the  results.  Results  are 
discussed  in  aggregate,  as  well  as  at  the  conclusion  of  this  section  of  the  report. 

Paired-sample  and  independent-sample  t  tests  were  used  in  the  analyses.  Directional  tests 
were  used  with  difference  scores.  The  hypotheses  tested  were  that  the  differences  between  post- 
and  pre-instruction  test- scores,  self-ratings,  and  percentages  would  be  greater  than  zero. 
Improvements  in  factual  knowledge,  application  of  that  knowledge,  and  performance  were 
expected  after  instruction  compared  to  before  instruction. 

Contextualized  Instruction:  Confined  Space  Entry  Courseware 

An  instructor  from  the  Occupational,  Safety  and  Health  Technology  Department  at 
Odessa  College,  Odessa,  developed  a  lesson  covering  the  procedures  for  confined  space  entry. 
The  objective  of  the  lesson  was  to  be  100%  knowledgeable  of  OSHA  regulations  for  confined 
space  entry.  The  lesson  had  seven  sections.  The  developer  eliminated  practice  from  the  lesson. 

Multimedia 

The  graphics  used  in  the  lesson  were  created  by  the  instructor  and  student  assistant.  A 
scanner  was  used  to  create  graphics  of  forms  and  permits  required  for  the  procedure.  A  digital 
camera  was  used  to  capture  actors  going  through  safety  procedures  for  confined  space  entry.  The 
actors  were  students  enrolled  in  the  occupational  safety  and  health  program.  The  graphics  were 
intended  to  increase  lesson  relevance  by  showing  students  using  equipment  at  the  site  where  they 
would  be  performing  their  hands-on  training.  The  relevance  of  the  graphics  was  expected  to 
contribute  to  student  self-efficacy  when  performing  the  safety  procedures.  However,  a  self- 
efficacy  measure  was  not  collected  during  hands-on  training. 

Computer  Equipment  and  Implementation 

A  computer  lab  in  the  Continuing  Education  Building  was  the  site  of  the  evaluation.  The 
XAIDA  Deliver  software  and  lesson  files  were  loaded  on  a  server.  The  software  was  then  copied 
onto  hard  drives  of  12  Pentiums.  The  VGA  monitors  were  only  capable  of  handling  16  colors. 
This  lead  to  a  color  distortion  in  the  majority  of  graphics,  which  were  24-bit  true  color,  presented 
in  the  lesson.  Color  distortion  was  worse  on  some  monitors  than  on  other  monitors.  The  monitor 
display  areas  were  set  at  640  x  480. 

Participants 

Two  night  classes,  totaling  14  students,  participated  in  the  classroom  evaluation  of  the 
Confined  Space  Entry  courseware.  Data  from  two  students  were  dropped  from  the  evaluation  due 
to  incomplete  posttests. 
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Evaluation  Procedures 

Students  came  to  class  and  were  instructed  to  report  to  the  Continuing  Education 
Building  for  a  computer-based  lesson  on  confined  space  entry.  Before  beginning  the  lesson 
students  completed  the  pre-measures  of  computer  comfort  and  a  19-item,  matching,  true/false, 
and  multiple-choice  pretest.  Students  entered  their  lesson  start  times  before  beginning  the  lesson. 
Written  and  verbal  instructions  were  provided  on  how  to  use  the  interface.  The  instructor  and 
researcher  were  available  to  answer  questions.  No  practice  exercises  were  presented  on  the 
computer.  At  the  end  of  the  lesson,  students  entered  their  stop  times  and  completed  a  19-item 
posttest,  9-item  Training  Assessment  Survey  (TAS),  and  instructional  preference  item.  Students 
put  written  comments  on  the  back  of  their  test  booklets. 

Results 

Table  2  contains  the  mean  results  from  the  classroom  evaluation.  The  lesson  produced  a 
statistically  significant  increase  in  test  scores  (t(12)  =  2.6,  p  <  .01).  No  difference  was  found  in 
self-ratings  of  comfort  using  computers.  On  average  students  rated  themselves  “comfortable” 
using  computers.  Over  half  of  the  students  showed  a  preference  for  receiving  instruction  on 
confined  space  entry  as  self-paced  CBT,  rather  than  as  classroom  lecture. 


Table  2.  Mean  Results  from  Evaluating  the  Confined  Space  Entry  Courseware 


19-point 

Pretest 

score 

19-point 

Posttest 

score 

Knowledge 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7- very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7-  very 

Post-measure 

instructional 

preference 

Time  on  task 
in  minutes 
(n  =  12) 

13.5 

(sd  =  2.0) 

15.1 

(sd  =  2.1) 

1.6 

(e  <  -01) 

5.4 

(sd  =  2.3) 

5.6 

(sd=  1.8) 

.20 

(2  =  NS) 

58%  CBT 

42%  Lecture 

30.3 

(range  15-50) 

Training  Assessment  Survey:  Confinded  Space  Entry 


Ite  m  s 


Figure  1.  Training  Assessment  Survey  Results  for  Confined  Space  Entry  Courseware 
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Results  from  the  training  assessment  survey  are  presented  in  Figure  1 .  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 

Overall  the  courseware  received  relatively  high  ratings  from  the  students.  The  item  that 
measures  lesson  relevance  is  critical  in  validating  the  ratings  for  the  remaining  items  on  the 
survey.  Students  rated  lesson  relevance  sufficiently  high  (M  =  6.4)  to  validate  the  other  ratings. 
Easy  to  understand  content  (M  =  4.1)  was  the  only  item  to  receive  an  average  rating  below  “5.” 

Discussion 

The  Confined  Space  Entry  courseware  was  effective  in  producing  a  12%  increase  in 
student  written  test  scores.  The  courseware  was  very  efficient,  comparing  average  time  to 
complete  the  module  to  nominal  class  time  of  50  minutes.  Allowing  students  to  proceed  at  their 
own  pace  saved  on  average  40%  of  classroom  time.  The  time  saved  could  be  used  to  extend 
hands-on  practice  with  the  safety  procedure.  Hands-on  practice  could  also  be  staggered  to  allow 
students  more  time  with  the  safety  equipment. 

The  majority  of  students  completing  the  courseware  expressed  a  preference  for  the  lesson 
as  self-paced  CBT  over  group-paced  lecture.  Students  provided  input  to  improve  the  courseware. 
They  suggested  that  titles  describing  series  of  steps  in  the  procedure  be  more  explicit.  Their 
suggestion  was  to  use  more  informative  titles  (i.e.,  Obtain  the  permit.  Post  the  permit,  etc.), 
rather  than  numbered  steps  (i.e.  Step  1 ,  Step  2,  etc).  It  is  likely  that  the  low  rating  for  easy  to 
understand  lesson  content  was  driven  by  a  potential  for  confusion  from  the  numeric  titles. 

An  evaluation  of  the  effect  of  contextualized  instruction  is  needed  to  determine  if 
contextualization  increases  learning  potential  of  courseware.  Contextualized  instruction  has  the 
potential  to  reduce  hands-on  training  time  and  enhance  transfer  of  training.  An  evaluation  could 
involve  comparing  the  Confined  Space  Entry  courseware  to  available  CBT,  or  modifying  the 
current  courseware  by  removing  context  relevant  text  and  multimedia.  An  advantage  of  XAIDA 
courseware  is  that  it  can  be  readily  modified  and  customized  to  meet  instructional  needs. 

Impact  of  ICW  on  Task  Performance 
Bank  Reconciliation  Courseware 

An  instructor  from  the  Marketing  and  Management  Department  at  South  Plains  College, 
in  Lubbock,  developed  a  lesson  covering  the  procedures  for  reconciling  a  bank  statement  The 
objective  of  the  lesson  was  to  be  able  to  identify  the  steps  used  to  reconcile  the  bank  balance  and 
book  balance;  and  be  able  to  perform  calculations  used  in  reconciliation.  The  lesson  had  two  tiers 
with  four  sections  each.  The  students  were  expected  to  stay  in  the  practice  exercises  until  they 
mastered  100%  of  the  facts. 
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Multimedia 

The  instructor  used  a  scanner  to  create  attention-getting  and  motivational  graphics  used  in 
the  lesson  introduction.  A  series  of  graphics  that  represented  changes  in  the  book  and  bank 
statements  were  created  in  Paint®  using  colored  text  for  emphasis. 

Computer  Equipment  and  Implementation 

A  computer  lab  in  the  same  building  as  the  classroom  was  the  site  for  the  evaluation.  The 
lab  contained  thirty  Pentium,  200  MHz  computers  and  an  instructor-computer  attached  to  a 
projector.  The  instructor-computer  was  used  to  demonstrate  interface  features  that  the  students 
would  be  using  to  navigate  the  lesson  and  practice  exercises. 

A  computer  lab  assistant  loaded  the  XAIDA  Deliver  software  the  day  before  the 
evaluation.  The  researcher  and  instructor  loaded  the  lesson  files  the  day  of  the  evaluation.  The 
software  had  to  be  loaded  on  a  per  computer  basis  because  the  lab  was  not  structured  to  allow 
software  to  be  passed  across  the  network.  The  monitor  display  areas  were  set  at  640  x  480  and 
color  pallets  at  256  colors. 

Participants 

Two  classes,  totaling  23  students,  participated  in  the  classroom  evaluation  of  the  Bank 
Reconciliation  lesson.  Fifteen  students  were  enrolled  in  Basic  Accounting  and  eight  students 
were  enrolled  in  Introduction  to  Management. 

Evaluation  Procedures 

Students  reported  to  the  computer  lab  at  their  regularly  scheduled  class  times.  Both 
classes  were  50  minutes  in  duration.  Before  beginning  the  lesson  students  completed  the  pre¬ 
measures  of  instructional  preference  (CBT  or  traditional  lecture)  computer  comfort  and  a  14- 
item,  multiple-choice  pretest.  Half  of  the  items  on  the  pretest  and  posttest  measured  factual 
knowledge,  while  the  other  half  measured  applied  knowledge.  The  applied  knowledge  items 
required  that  the  students  calculate  adjusted  book  balance,  and  additions  and  deductions  from  the 
bank  balance. 

Students  entered  their  lesson  start  times  before  beginning  the  lesson.  The  instructor 
demonstrated  how  to  use  the  interface  features  using  the  instructor-computer  and  projector.  At 
the  end  of  the  lesson  and  practice,  students  entered  their  stop  times,  rated  their  comfort  using 
computers,  completed  a  14-item  posttest,  1 1 -item  training  assessment  survey,  and  post-measure 
of  instructional  preference.  Students  put  written  comments  on  the  back  of  their  evaluation 
booklets. 

A  problem  arose  in  the  practice  exercises.  A  true/false  item  was  discovered  to  give 
erroneous  feedback.  Between  classes  the  lesson  was  modified  to  correct  the  error,  although  the 
error  persisted  with  the  second  class.  The  error  was  later  found  to  be  due  to  a  corrupt  database. 
Students  had  been  told  to  ignore  the  question  completely  by  using  the  Skip  Question  button. 
Nevertheless,  the  practice  item  may  have  had  a  negative  impact  on  learning. 
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Results 

Table  3  contains  the  mean  results  from  the  classroom  evaluation. 


Table  3.  Mean  Results  from  Evaluating  the  Bank  Reconciliation  Courseware 


14-point 

Pretest 

score 

14-point 

Posttest 

score 

Knowledge/ 
Performance 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  ail, 
7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7- very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  =  19) 

7.7 

(sd  =  2.2) 

9.0 

(sd  =  2.6) 

1.3 

(g  < .005) 

5.41 

(sd  =  1.3) 

5.45 

(sd=  1.2) 

.04 

(g  =  NS) 

Pre-measure 
48%  CBT 

52%  Lecture 
Post-measure 
52%  CBT 

48%  Lecture 

16.1 

(range  12-25) 

The  lesson  produced  a  statistically  significant  increase  in  overall  test  scores  (t(22)  =  2.8,  g 
<  .005).  The  significant  gain  score  was  driven  by  an  increase  in  scores  on  the  seven  factual 
knowledge  items  (Mpretest  =  3.3,  Mposttest =  4.5;  t  (22)  =  2.8,  g  <  .005),  not  the  seven  applied 
knowledge  items  (Mprctest  =  4.4,  Mposttest =  4.5;  g  =  NS).  No  difference  was  found  in  self-ratings 
of  comfort  using  computers.  Students  were  initially  comfortable  using  computers  (M  =  5.4). 
Student  instructional  preferences  changed  after  experiencing  the  computer-based  lesson 
compared  to  before  the  experience.  More  students  preferred  that  the  lesson  on  bank 
reconciliation  be  presented  as  CBT  than  group  lecture  after  they  experienced  the  courseware. 

Results  from  the  training  assessment  survey  are  presented  in  Figure  2.  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 

Overall  the  courseware  received  moderate  ratings  from  the  students.  The  item  that 
measures  lesson  relevance  is  critical  in  validating  the  other  ratings.  Students  rated  lesson 
relevance  sufficiently  high  (M  =  5.6).  Six  other  items  were  rated  below  the  acceptable  level.  The 
courseware  failed  to  keep  students’  attention  (M  =  4.5),  lesson  content  was  not  easy  to 
understand  (M  =  4.4),  practice  was  less  than  adequate  (M  =  4.6)  and  repetition  failed  to  stimulate 
the  students  (M  =  4.9).  Taken  together  the  lesson  failed  to  motivate  (M  =  4.7)  the  students. 
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Training  Assessment  Survey:  Bank  Reconciliation 


Figure  2.  Training  Assessment  Survey  Results  for  the  Bank  Reconciliation  Courseware 


Discussion 

The  Bank  Reconciliation  courseware  produced  a  17%  increase  in  written  test  scores. 
However,  the  courseware  failed  to  improve  students’  abilities  calculating  bank  and  book 
balances.  There  were  two  factors  that  can  account  for  poor  task  performance.  One  factor  is  the 
feedback  error  found  in  the  practice  exercises.  Although,  students  were  told  to  skip  the  question 
containing  the  error,  the  faulty  feedback  likely  had  a  negative  impact  on  learning.  Feedback  is  a 
very  powerful  learning  event.  Time  is  the  other  factor  that  may  explain  poor  performance. 
Students  had  only  40  to  45  minutes  to  learn  the  interface,  complete  the  evaluation  instruments, 
and  complete  the  courseware.  Undoubtedly,  time  pressure  counteracts  the  learning  potential  of 
self-paced  CBT. 

More  students  chose  to  receive  the  bank  reconciliation  lesson  as  self-paced  CBT  after 
their  experience  compared  to  before.  Student  responses  to  the  training  assessment  survey  should 
be  addressed  to  improve  the  courseware,  given  the  students  expressed  interest.  The  instructor  can 
easily  make  improvements  and  modifications,  using  student  input  as  a  guide,  to  the  courseware. 
Programmers  have  already  corrected  the  bug  found  in  the  feedback  generated  in  practice.  Once 
improvements  have  been  made,  the  new  courseware  is  expected  to  produce  additional  increases 
in  written  test  scores,  improved  task  performance,  and  a  positive  assessment  from  the  students. 

The  findings  from  this  evaluation  suggest  that  class  time  needs  to  be  extended  for  the 
purpose  of  evaluating  new  courseware.  Night  classes  appear  to  be  a  good  testing  ground  for  self- 
paced  CBT  because  night  classes  are  normally  longer  than  50  minutes. 

Simpson  Analog  Multimeter  Courseware 

An  instructor  from  the  Multimodal  Transportation  Department  at  St.  Philip’s  College, 
Southwest  Campus,  in  San  Antonio,  developed  a  lesson  covering  the  setup  and  use  of  a  Simpson 
Analog  Multimeter.  The  lesson  objectives  included:  possess  the  ability  to  setup  and  use  the 
meter,  name  the  parts  of  the  meter,  and  identify  the  function  of  each  part.  The  lesson  had  seven 
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sections.  The  students  were  expected  to  stay  in  the  practice  exercises  until  they  mastered  1 00%  of 
the  material. 

Multimedia 

The  instructor  used  a  scanner  and  digital  camera  to  create  the  graphics  used  in  the  lesson. 
The  instructor  used  Windows  Sound  Recorder  application  to  create  audio  narration.  Several 
graphic  and  audio  files  were  combined  using  Adobe  Premiere®  to  create  video  resources  for  the 
lesson.  The  quality  of  the  audio  was  poor  during  the  evaluation,  for  an  unknown  reason. 

The  use  of  multimedia  in  the  lesson  was  inconsistent.  At  different  points  in  the  lesson, 
resources  were  presented  as  static  graphics  with  narration  and  redundant  on-screen  text,  others  as 
narration  and  redundant  on-screen  text,  and  others  as  text  only. 

Computer  Equipment  and  Implementation 

A  computer  lab  in  a  building  near  the  classroom  was  the  site  for  the  evaluation.  Computer 
lab  support  personnel  loaded  the  XAID  A  software  and  lesson  files  from  a  server  on  ten  Pentium, 
120  MHz  computers  with  sound  cards  and  an  instructor’s  computer  attached  to  a  projector.  Each 
computer  was  equipped  with  a  headset.  The  monitor  display  areas  were  set  at  640  x  480  and 
color  pallets  at  256  colors.  The  instructor-computer  was  used  to  demonstrate  interface  features 
that  the  students  would  be  using  to  navigate  the  lesson. 

Participants 

Eight  students  enrolled  in  an  aircraft  electrical  systems  class  participated  in  the  classroom 
evaluation  of  the  Simpson  Analog  Multimeter  lesson.  Prior  to  participating  none  of  the  students 
reported  being  “familiar”  with  the  meter. 

Evaluation  Procedures 

Students  reported  to  the  computer  lab  from  their  regularly  scheduled  classroom.  Before 
beginning  the  lesson  students  completed  the  pre-measures  of  instructional  preference  (CBT  or 
traditional  lecture)  computer  comfort,  familiarity  with  the  meter,  and  a  14-item,  fill-in-the-blank 
and  matching  pretest.  Half  of  the  items  on  the  pretest  and  posttest  tapped  into  students’ 
knowledge  of  part  names  and  the  other  half  tapped  into  students’  knowledge  of  each  part’s 
function. 

The  instructor  demonstrated  the  XAID  A  interface  features  using  the  instructor’s  computer 
and  projector.  At  the  end  of  the  lesson,  students  completed  a  14-item  posttest,  12-item  training 
assessment  survey,  and  post-measure  of  instructional  preference.  Students  put  written  comments 
on  the  back  of  their  evaluation  booklets.  Following  completion  of  the  booklet,  students  returned 
to  their  normal  classroom  for  a  performance  task.  Students  were  required  to  setup  the  meter  and 
measure  both  resistance  and  voltage  from  a  circuit  board.  Six  written  responses  were  required  for 
each  measurement  type.  Error  rates  were  tallied  across  the  12  items. 
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Results 

Table  4  contains  the  mean  results  from  the  classroom  evaluation.  The  lesson  produced  a 
statistically  significant  increase  in  overall  test  scores  (t(7)  =  6.4,  p  <  .0001).  The  significant  gain 
score  was  driven  by  an  increase  in  scores  on  the  seven  naming  items  (Mpretest =  1  - 1 ,  Mp0Sttest = 
6.8;  t  (7)  =  21.4,  p  <  .0001),  not  the  seven  functional  items  (Mpretest  =  4.5,  Mposttest  =  5.8;  p  = 
NS).  No  statistical  difference  was  found  in  self-ratings  of  comfort  using  computers.  After  the 
lesson,  50%  of  the  class  categorized  themselves  as  being  at  least  “familiar”  with  the  multimeter. 
Initially  no  student  indicated  being  “familiar”  with  the  meter. 


Table  4.  Mean  Results  from  Evaluating  the  Simpson  Analog  Multimeter  Courseware 


14-point 

Pretest 

score 

14-point 

Posttest 

score 

Knowledge/ 
Performance 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 

7-  very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  =  8) 

5.6 

(sd=  1.9) 

12.5 

(sd  =  2.0) 

6.9 

(p<.0001) 

4.5 

(sd  =  1.3) 

4.0 

(sd=  1.6) 

-.5 

(b=ns) 

Pre-measure 

33.3%  CBT 
33.3%  Lecture 
33.3%  CAI 
Post-measure 
50%  CBT 

25%  Lecture 
25%  CAI 

30.5 

(range  19-40) 

On  average  students  committed  .8  errors  measuring  resistance  and  1.3  errors  measuring 
volts.  Fifty  percent  of  the  students  performed  both  measures  without  error.  A  common  error 
generated  by  students  was  failing  to  shut  off  power  and  touch  leads  together  to  adjust  the  meter 
to  zero,  prior  to  taking  a  resistance  reading. 

Student  instructional  preferences  were  equally  dispersed  among  CBT,  lecture,  and  CAI 
before  experiencing  the  courseware.  Fifty  percent  of  the  students  preferred  the  lesson  on  the 
Simpson  Analog  Multimeter  as  self-paced  CBT  after  they  had  the  XAIDA  experience.  The 
remainder  of  student  preferences  was  evenly  split  between  lecture  and  CAI. 

Results  from  the  training  assessment  survey  are  presented  in  Figure  3.  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 
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Training  Assessment  Survey:  Simpson  Analog  Multimeter 


Items 


Figure  3.  Training  Assessment  Survey  Results  for  the  Simpson  Multimeter  Courseware 


Overall  the  courseware  received  low  ratings  from  the  students.  The  item  that  measures 
lesson  relevance  is  critical  in  validating  the  remaining  ratings.  Students  rated  lesson  relevance 
sufficiently  high  (M  =  5.3)  to  validate  the  other  ratings.  Eight  of  the  12  items  were  rated  below  an 
acceptable  level.  The  courseware  failed  to  keep  students  attention  (M  =  3.4),  lesson  content  was 
not  easy  to  understand  (M  =  4.5),  practice  was  less  than  adequate  (M  =  4.1),  terms  were  not 
effectively  emphasized  (M  =  4.8),  repetition  failed  to  stimulate  the  students  (M  =  3.4), 
multimedia  failed  to  add  to  learning  (M  =  4.3),  and  personal  interactions  were  less  than  sufficient 
(M  =  4.9).  Taken  together  the  lesson  failed  to  motivate  (M  =  4.4)  the  students. 

Discussion 

The  Simpson  Analog  Multimeter  courseware  did  produce  123%  increase  in  average 
knowledge  gain  scores.  The  courseware  was  very  efficient,  comparing  average  time  of  30.5 
minutes  to  complete  the  module  to  nominal  class  time  of  50  minutes.  Allowing  students  to 
proceed  at  their  own  pace  saved  on  average  40%  of  classroom  time. 

Students  did  not  report  having  a  completely  positive  experience  with  the  courseware,  nor 
did  they  report  being  comfortable  learning  from  computers,  although  the  experience  swayed 
students’  instructional  preference  toward  self-paced  CBT.  Afterward,  half  of  the  students  would 
prefer  to  receive  instruction  on  the  Simpson  Analog  Multimeter  as  self-paced  CBT;  whereas 
before  only  one-third  of  the  students  were  interested  in  receiving  instruction  as  self-paced  CBT. 
Even  though  students  assessed  their  lesson  as  unmotivating,  less  than  adequate  in  keeping  their 
attention  and  providing  practice,  more  difficult  to  understand  than  they  had  hoped,  and  found  the 
multimedia  to  distract  from  learning;  they  managed  to  increase  their  average  test  scores  by  123%. 
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The  results  from  the  evaluation  suggest  that  research  is  needed  to  better  understand  how 
students  learn  from  computers.  Potentially,  perceived  deficiencies  in  the  courseware  are  an 
artifact  of  students  not  being  skilled  at  using  computers  as  learning  tools. 

Linear  Equations  in  Slope-Intercept  Form  Courseware 

An  instructor  from  the  Math  Department  at  Navarro  College,  in  Corsicana,  developed  a 
lesson  covering  linear  equations  in  slope-intercept  form.  The  objective  of  the  lesson  was  to  teach 
students  how  to  convert  an  equation  into  slope-intercept  form  and  use  it  to  graph  a  line.  The 
lesson  was  intended  to  help  students  prepare  for  the  TASP  test  by  providing  a  platform  to 
practice  matching  equations  to  graphs.  Students  were  expected  to  continue  practicing  until  they 
mastered  100%  of  the  material. 

Multimedia 

The  instructor  reused  a  basic  graphic  from  the  lesson  developed  the  previous  year.  The 
instructor  created  video  files  using  Adobe  Premiere®  by  interleaving  audio  (WAV)  and  graphic 
(BMP)  files.  The  video  files  produced  the  effect  of  an  animated  white  board. 

The  instructor  originally  created  the  audio  files  using  Windows  95  Sound  Recorder. 
However,  the  poor  sound  quality  was  improved  upon  by  re-recording  the  audio  at  Mei 
Technology  Corporation. 

Computer  Equipment  and  Implementation 

A  computer  lab  in  the  same  building  as  the  classroom  was  the  site  for  the  evaluation.  The 
lab  contained  20  Pentium  266  MHz  towers  with  sound  cards.  Each  computer  was  equipped  with 
a  headset.  The  length  of  the  headset  cords  was  incompatible  with  the  physical  layout  of  the 
computer  lab.  Use  of  the  headsets  was  uncomfortable  for  the  students,  as  it  did  not  allow  for 
freedom  of  movement. 

The  researcher  loaded  the  XAIDA  Deliver  software  and  courseware  files  in  the  lab  prior 
to  the  evaluation.  Two  video  files  had  to  be  omitted  from  the  lesson  because  the  computers 
lacked  the  software  to  decompress  the  files.  The  software  had  to  loaded  on  a  per  computer  basis, 
as  the  lab  was  not  structured  to  allow  software  to  be  passed  across  a  network.  Also,  the  software 
application  that  “write  protects”  the  hard  drive  had  to  be  disabled  during  the  evaluation. 

Participants 

A  class  of  nine  students  participated  in  the  evaluation  of  the  Linear  Equation  in  Slope- 
intercept  Form  courseware.  The  students  were  enrolled  in  the  second  summer  session  Math 
Seminar  for  TASP. 

Evaluation  Procedures 

Students  reported  to  the  computer  lab  instead  of  their  classroom  at  their  regularly 
scheduled  class  time.  Before  beginning  the  lesson  students  completed  the  pre-measures  of 
instructional  preference,  computer  comfort,  familiarity  with  the  lesson  topic,  and  a  15-item, 
multiple-choice  pretest.  Items  on  the  pretest  and  posttest  measured  factual  knowledge  and 
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applied  knowledge  (e.g.,  converting  equations  into  slope-intercept  form,  matching  equations  to 
graphs).  Applied  items  were  similar  to  those  found  on  the  TASP  test. 

Verbal  and  written  instructions  were  provided  on  use  of  the  interfaces.  At  the  end  of  the 
lesson  and  practice,  students  completed  a  15 -item  posttest,  12-item  training  assessment  survey, 
and  post-preference  item.  Students  put  written  comments  on  the  back  of  their  booklets. 

Results 

Table  5  contains  the  mean  results  from  the  classroom  evaluation.  The  lesson  produced  a 
statistically  significant  gain  score  (t(9)  =  6.1,  p  <  .0001).  A  significant  difference  was  also  found 
in  self-ratings  of  familiarity  with  the  topic  of  linear  equations  in  slope-intercept  form  (M  pre=  2.9, 
M  post =  4.0;  t(8)  =  2.0, 'p  <  .04).  Increases  in  student  comfort  using  computers  for  learning 
approached  significance  (M  pre=  5.0,  M  post=  5.8;  t(8)  =  1.7,  g  <  .07).  Familiarity  and  comfort 
rating  were  made  on  a  7-point  scale  where  larger  numbers  represented  more  of  the  characteristic. 

Student  instructional  preferences  changed  after  experiencing  the  computer-based  lesson 
compared  to  before  the  experience.  More  students  preferred  the  lesson  on  linear  equations  in 
slope-intercept  form  be  presented  as  instructor-paced  CAI  (44%)  than  as  either  self-paced  CBT 
(33%)  or  group-paced  lecture  (22%),  after  they  had  the  XAIDA  experience. 


Table  5.  Mean  Results  from  Evaluating  the  Slope-Intercept  Form  Courseware 


14-point 

Pretest 

score 

14-point 

Posttest 

score 

Knowledge/ 
Performance 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7- very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7- very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  =  9) 

4.9 

(sd  =  3.5) 

9.8 

(sd  =  3.5) 

4.9 

(p<.0001) 

5.0 

(sd=  1.7) 

5.8 

(sd=  1.4) 

.8 

(R=NS) 

Pre-measure 

33.3%  CBT 
33.3%  Lecture 
33.3%  CAI 
Post-measure 
33.3%  CBT 
22.2%  Lecture 
44.4%  CAI 

26.7 

(range  16-41) 

Results  from  the  training  assessment  survey  are  presented  below  in  Figure  4.  Students 
rated  the  survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors 
anchored  the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 

Overall  the  courseware  received  acceptable  ratings  from  the  students.  The  item  that 
measures  lesson  relevance  is  critical  in  validating  the  remaining  ratings.  Students  rated  lesson 
relevance  sufficiently  high  (M  =  5.7)  for  validation  purposes.  The  amount  of  repetition  in  the 
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lesson  was  less  than  stimulating  to  the  students  (M  =  3.9).  This  repetition  and  the  motivational 
capacity  of  the  lesson  (M  =  4.7)  were  the  only  items  to  receive  average  ratings  below  “5.” 


Training  Assessment  Survey:  Linear  Equations  in  Slope-Intercept  Form 


Items 


Figure  4.  Training  Assessment  Survey  Results  for  Slope-Intercept  Form  Courseware 


Discussion 

The  Linear  Equations  in  Slope-Intercept  Form  courseware  produced  a  100%  increase  in 
performance  scores.  The  courseware  was  very  efficient,  comparing  average  time  of  26.7  minutes 
to  complete  the  module  to  nominal  class  time  of  50  minutes.  Allowing  students  to  proceed  at 
their  own  pace  saved  on  average  nearly  50%  of  classroom  time. 

The  students  liked  the  idea  of  learning  from  computers,  although  they  would  rather  the 
instructor  maintain  control  over  what  information  is  presented  when.  Interactivity  that  is  built 
into  the  courseware  making  it  self-paced,  requires  the  student  take  responsibility  for  what 
information  is  presented  when.  This  feature  may  demotivate  students  who  require  more  guidance 
in  learning. 


ICW  Implemented  as  CAI:  Oral  Medication  Administration  Courseware 

The  LVN  program  director  at  South  Texas  Junior  College,  in  Uvalde,  developed  a  lesson 
covering  the  administration  of  oral  medication.  The  objective  of  the  lesson  was  to  be  able  to 
identify  the  five  steps  used  in  the  administration  of  oral  medications.  The  lesson  had  five 
sections. 

The  class  had  received  a  lecture  on  oral  medication  administration  earlier  in  the  semester. 
The  instructor,  therefore,  decided  to  present  the  lesson  as  computer-assisted  instruction  (CAI). 
The  instructor  controlled  the  pace  of  the  lesson  by  presenting  it  on  a  large  screen  at  the  front  of 
the  classroom.  The  evaluation  was  the  first  implementation  of  an  XAIDA  lesson  as  CAI. 


15 


Multimedia 


The  instructor  used  a  scanner  and  Paint®  to  create  the  graphical  resources  used  in  the 
lesson.  An  instructional  technologist  at  St  Philip’s  College  captured  the  video  files  used  in  the 
lesson  from  a  nursing  videotape. 

Computer  Equipment  and  Implementation 

A  Pentium  233  MHz  MMX  Dell  computer  attached  to  a  projector  system  served  as  the 
instructional  platform.  The  classroom  was  equipped  with  a  pull-down  screen.  XAIDA  Deliver 
software  was  loaded  onto  the  computer  hard  drive.  The  lesson  ran  off  of  a  CD. 

Participants 

Twenty-nine  students  enrolled  in  the  Advanced  Nursing  Skills  course  participated  in  the 
classroom  evaluation  of  the  Administration  of  Oral  Medication  lesson.  The  LVN  program 
requires  that  students  have  20  hours  of  instruction  on  the  computer.  Participation  counted  toward 
that  requirement. 

Evaluation  Procedures 

Students  reported  to  class  at  their  regularly  scheduled  time.  Before  beginning  the  lesson 
students  completed  pre-measures  of  instructional  preference  (self-paced  CBT,  traditional  group- 
paced  lecture,  or  instructor-paced  CAT)  computer  comfort  and  a  10-item,  multiple-choice  pretest. 

The  instructor  presented  the  lesson  at  the  front  of  the  class.  Class  participation  was 
encouraged  throughout  presentation  of  lesson  content.  A  response  sheet  for  practice  items  was 
included  in  the  test  booklet.  Students  initially  provided  written  responses  to  the  practice  items 
followed  by  verbal  responding.  However,  written  responses  were  replaced  with  Vs  after  several 
multiple-choice  and  fill-in-the-blank  exercises  required  up  to  five  responses.  Multiple  responses 
were  timely  and  cumbersome  to  write  in  the  space  provided.  A  V  represented  that  the  student 
knew  the  answer.  Following  the  practice  exercises,  post-measures  of  computer  comfort,  posttest, 
and  12-items  training  assessment  survey  with  post-preference  measure  were  administered. 

Results 

Table  6  contains  the  mean  results  from  the  classroom  evaluation.  The  lesson  produced  a 
statistically  significant  but  small  increase  of  .7  points  in  test  scores  (t(28)  =  3.3,  g  <  .001).  The 
small  gain  score  can  be  explained  as  a  “ceiling  effect”  given  that  the  average  pretest  score  of  8.8 
does  not  leave  much  room  for  improvement.  Students  rated  their  comfort  using  computers  higher 
after  experiencing  CAL  No  real  change  was  found  in  instructional  preferences  after  versus  before 
experiencing  CAL  Nearly  80%  of  the  class  showed  a  preference  for  learning  with  computer- 
based  instruction. 

Results  from  the  training  assessment  survey  are  presented  in  Figure  5.  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  ratings  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 
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Table  6.  Mean  Results  from  Evaluating  the  Oral  Medication  Administration  Courseware 


10-point 
Pretest  score 

10-point 
Posttest  score 

Knowledge 

Gain  score 

Pre-rating  of 
comfort  using 
computers 

1-not  at  all, 

7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Change  in 
comfort  rating 
using  computers 

1-  not  at  all, 

7-  very 

Instructional 

preference 

8.8 

(sd  =  .99) 

9.5 

(sd  -  .69) 

.7 

(E  <  .001) 

4.7 

(sd=  1.9) 

5.2 

(sd=  1.7) 

.37 

(p  <  .03) 

Pre-measure 

53%  CBT 

17%  Lecture 
29%  CAI 
Post-measure 
55%  CBT 

17%  Lecture 
28%  CAI 

Training  Assessment  Survey:  Oral  Medication  Administration 


Items 


Figure  5.  Training  Assessment  Survey  Results  for  Oral  Medication  Courseware 


The  item  that  measures  lesson  relevance  is  critical  to  the  validity  of  the  remaining  survey 
items.  Students  rated  lesson  relevance  sufficiently  high  (M  =  6.2)  for  purposes  of  validation. 
Overall  the  courseware  received  acceptable  ratings  from  the  students,  with  the  exception  of  the 
repetitiveness.  The  amount  of  repetition  in  the  lesson  was  less  than  stimulating  to  the  students  (M 
-  3.1).  The  source  of  the  repetitiveness  is  most  likely  found  in  the  practice  exercises.  Practice  is 
designed  to  drill  the  students  until  they  possess  mastery  of  the  facts  presented  in  the  lesson. 
Students  perceived  the  lesson  as  motivating  (M  =  5.1),  although  the  instructional  sequence  was 
not  able  to  keep  students’  full  attention  (M  =  4.9). 
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Discussion 

The  Oral  Medication  Administration  courseware  is  the  first  effort  at  implementing 
XAIDA  as  computer-assisted  instruction.  The  courseware  produced  an  8%  increase  in  knowledge 
gain  scores.  The  percent  gained  would  likely  have  been  higher  had  the  class  not  previously  been 
lectured  on  the  lesson  topic.  Regardless,  the  students  reported  having  an  overall  positive 
experience  with  CAL  The  class  as  a  whole  expressed  interested  in  computer-based  instruction. 

Training  Faculty  and  Staff  Using  ICW:  Linking  Files  to  a  Webpage  Courseware 

A  technology  communications  instructor  at  Tomball  College,  in  Tomball,  developed  a 
lesson  covering  the  steps  to  linking  files  to  a  WebPage  using  Netscape  Composer®.  The 
objective  of  the  lesson  was  to  understand  how  to  use  features  of  Netscape  Composer®  to  link 
files  to  a  Webpage.  The  lesson  had  five  sections. 

Multimedia 

The  instructor  used  screen  captures  of  the  Netscape  Explorer®  interface  imported  into 
Paint®  to  create  the  graphical  resources  used  in  the  lesson.  The  instructor  created  audio  narration 
to  accompany  the  graphical  representations  of  the  procedure.  Music  and  audio  narration  were 
used  to  gain  students’  attention  in  the  lesson  introduction  and  throughout  the  lesson.  However, 
the  use  of  multimedia  was  inconsistent. 

Computer  Equipment  and  Implementation 

A  computer  lab  with  Pentium  90  MHz  computers  was  the  site  of  the  evaluation. 

Computer  lab  personnel  loaded  the  XAIDA  Deliver  software  from  a  server  onto  ten  computers. 
The  researcher  loaded  the  lesson  files  onto  each  computer.  Verbal  instructions  on  using  the 
XAIDA  interfaces  were  provided  individually  to  each  participant. 

Participants 

Eight  staff  and  faculty  at  Tomball  College  volunteered  to  participate  in  the  evaluation. 
Fifty  percent  of  participants  reported  that  they  had  no  ability  using  Netscape  Composer®  to  link 
files  to  WebPages.  Approximately  37%  of  participants  reported  having  yhz>  to  good  ability. 

Evaluation  Procedures 

Participants  came  into  the  computer  lab  within  a  designated  time  slot.  Each  individual 
was  given  an  evaluation  booklet  and  verbal  instructions  on  how  to  begin  the  lesson.  Before 
beginning  pre-measures  were  collected  on  self-ratings  of  computer  comfort,  knowledge  of  how 
to  link  files  using  Netscape  Composer®,  ability  to  link  files  using  Netscape  Composer®, 
instructional  preference,  and  a  10-point  multiple-choice  test.  Participants  remained  in  practice  at 
the  end  of  the  lesson  until  they  mastered  100%  of  the  facts.  The  posttest  and  training  assessment 
survey  with  a  post-measure  of  instructional  preference  was  administered  following  practice. 

Results 

Table  7  contains  the  mean  results  from  the  evaluation.  The  lesson  produced  a  statistically 
significant  increase  in  overall  scores  (t(7)  =  2.3,  p  <  .03).  The  computer  comfort  average  rating  of 
6.3  was  the  high-end  of  the  7-point  scale  and  showed  no  change  after  instruction.  However,  a 
significant  increase  was  found  after  instruction  (M  Pre=  3.0,  M  post  =4.4;  t(7)  =  2.3,  p  <  .03)  in 
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ratings  for  level  of  knowledge  about  how  to  link  files  using  Netscape  Composer®.  After 
instruction  the  average  rating  for  ability  to  link  files  using  Netscape  Composer®  was 
significantly  higher  than  the  average  rating  before  instruction  (M  pre  =  3 . 1 ,  M  post  =  4.9;  t(7)  =  2.4, 
E  <  -02).  The  7-point  ability  scale  was  labeled  with  no  ability  (1),  poor  ability  (3),  fair  ability 
(5),  and  good  ability  (7).  All  participants  reported  having  some  level  of  ability  using  Netscape 
Composer®  to  link  files  to  WebPages  following  instruction.  Of  those  participants  responding, 
approximately  63%  reported  their  ability  between  fair  and  good. 


Table  7.  Mean  Results  from  Evaluating  the  Linking  Files  to  WebPages  Courseware 


10-point 

Pretest 

score 

10-point 

Posttest 

score 

Knowledge 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7-  very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  —  6) 

6.4 

(sd=  1.9) 

7.9 

(sd=  1.4) 

1.5 

(p  <  .03) 

6.3 

(sd=  1.0) 

6.3 

(sd  =  1.0) 

0.0 

(E=NS) 

Pre-measure 
100%  CBT 
Post-measure 
62.5%  CBT 
12.5%  Lecture 
25.0%  CAI 

18.3 

(range  10-26) 

Results  from  the  training  assessment  survey  are  presented  in  Figure  6.  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points  (e.g.,  sufficient  and  insufficient,  adequate  and  inadequate).  Low  numbers 
signified  unacceptable  ratings  and  high  numbers  signified  acceptable  ratings.  Average  ratings 
below  “5”  indicate  aspects  of  the  courseware  that  need  improvement. 


Training  Assessm ent  Survey:  Linking  Files  to  WebPages 


.......  .  ,  v  ..  .. v  ; 


\  \  V  \ 

’-v  \  %  \  V 

%  \  \ 


Figure  6.  Training  Assessment  Survey  Results  for  Linking  Files  to  WebPages  Courseware 
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Overall  the  courseware  received  moderate  ratings  of  acceptability  from  the  users  The  item 
that  measures  lesson  relevance  is  critical  to  validating  the  ratings  for  the  remaining  survey  items 
Students  rated  lesson  relevance  in  the  acceptable  range  (M  =  5.4)  for  purposes  of  validation 

C°f ent, WSS  n0t  t0  undasland  =  4-6).  The  amount  of  repetition  in  the  lesson  was 
=  ?  t^stimulatin§  t0  the  users  (M  -  4.3)  and  the  use  of  multimedia  did  not  add  to  learning  (M 
j.y).  However,  users  perceived  the  lesson  as  motivating  (M  =  5.0). 

Discussion 


The  Linking  Files  to  a  Webpage  courseware  produced  a  23%  increase  in  faculty 
and  staffs  factual  knowledge  about  Netscape  Composer®.  The  courseware  also  produced 
increases  m  faculty  and  staff  s  self-perceived  levels  of  knowledge  and  ability  using  Netscape 
Composer®  to  link  files  to  Webpages. 

Results  from  the  training  assessment  survey  suggest  that  improvements  should  be 
made  to  the  courseware.  Practice  exercises  need  to  be  examined  and  modified  to  better  target  the 
objective  of  the  lesson.  The  multimedia  needs  to  be  examined  and  modified  to  better 
communicate  important  information  to  the  user. 

The  shift  in  users’  instructional  preference  from  100%  self-paced  CBT  to  63% 
suggests  that  students  require  a  guide  to  help  them  leam  Netscape  Composer®. 


Impact  of  Humor  on  Learning:  Installing  RAM  Courseware 

The  Computer  Information  Systems  Department  Head  at  Amarillo  College,  in  Amarillo 
developed  a  lesson  covering  the  steps  to  installing  Random  Access  Memory  (RAM).  The 
objective  of  the  lesson  was  to  be  able  to  identify  and  recall  the  critical  steps  to  installing  SIMM 
RAM  in  a  microcomputer,  and  be  able  to  recognize  computer  jargon  associated  with  the 
installation  process.  The  lesson  was  presented  in  four  sections. 

Two  courseware  modules,  humorous  and  non-humorous,  were  created  for  the  evaluation. 
They  differed  on  four  graphics.  The  instructional  content  of  the  courseware  was  held  constant. 
We  expected  the  students  receiving  the  humorous  courseware/graphics  to  have  a  more  positive 
learning  experience  than  the  students  receiving  the  non-humorous  courseware/graphics. 

XAIDA  courseware  can  be  readily  modified,  which  is  one  of  the  attractive  feature  of 
XAIDA  as  an  authoring  tool.  The  developer  merely  gave  identical  file  names  to  the  pairs  of 
humorous  and  non-humorous  graphics  in  order  to  create  the  two  distinct  lessons. 

Multimedia 

The  instructor  scanned  in  graphics  from  various  sources.  Mei  Technology  Corporation 
used  a  digital  camera  to  create  other  graphics  showing  a  person  installing  RAM,  an  actual  SIMM 
chip,  and  the  proper  manner  to  handle  the  chip. 
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Computer  Equipment  and  Implementation 

A  computer  lab  with  Pentium  90  MHz  computers  running  Windows  NT  was  the  site  of 
the  evaluation.  A  computer  with  a  projector  served  as  a  demonstration  platform.  Windows  NT 
crashed  before  the  second  group  received  a  demonstration  of  the  interface.  Only  verbal 
instructions  were  provided  to  the  second  group. 

Participants 

Twenty-five  students  enrolled  in  the  Computer  Concepts  course,  offered  during  the 
second  summer  session,  participated  in  the  evaluation. 

Evaluation  Procedures 

Participants  reported  to  their  regularly  scheduled  classroom.  The  class  was  split  into  two 
groups.  The  first  group  had  another  class  scheduled  immediately  following  their  Computer 
Concepts  class.  The  second  group  had  no  other  classes  scheduled  in  the  morning. 

After  the  class  was  split,  the  first  group  completed  the  pretest  portion  of  the  evaluation 
booklet  and  reported  to  the  computer  lab.  Participants  were  randomly  assigned  to  either  the 
humorous  or  non-humorous  courseware.  The  first  group  was  given  a  demonstration,  written  and 
verbal  instructions  on  how  to  begin  the  lesson  and  use  the  interfaces.  The  second  group  did  not 
get  a  demonstration  of  the  interfaces. 

Participants  completed  the  lesson  and  remained  in  practice  until  they  mastered  1 00%  of 
the  facts.  The  posttest  and  training  assessment  survey  was  administered  following  the  practice 
exercises.  An  item  measuring  the  level  of  humor  in  the  courseware  was  added  to  the  training 
assessment  survey  as  a  manipulation  check. 

A  retention  measure  was  collected  one  week  following  the  original  evaluation.  The 
measure  included  items  from  both  forms  of  the  original  tests  with  improved  distracters. 
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Results 


Table  8  contains  the  mean  results  from  the  classroom  evaluation.  The  humor 
manipulation  was  effective  (t  (23)  =  2.6,  p  <  .01).  The  group  that  took  the  humorous  lesson  (M  = 
5.3)  gave  it  a  significantly  higher  humor  rating  on  a  7-point  scale  than  the  group  that  took  the 
non-humorous  lesson  (M_=  3.5). 


Table  8.  Mean  Results  from  Evaluating  the  Installing  RAM  Courseware 


14-point 

Pretest 

score 

14-point 

Posttest 

score 

Knowledge 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7-  very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  =  22) 

5.5 

(sd  =  2.0) 

12.4 

(sd=  1.9) 

6.9 

(p<-0001) 

4.4 

(sd=  1.3) 

5.3 

(sd=  1.4) 

.90 

(p  <  .005) 

Pre-measure 

31.6 

(range  20-50) 

32%  CBT 

28%  Lecture 
40%  CAI 
Post-measure 

68%  CBT 

12%  Lecture 
20%  CAI 

The  lesson  produced  a  statistically  significant  increase  in  overall  test  scores  (t  (24)  =  16.2, 
E  <  -0001)  on  the  14-item  written  test.  Statistical  differences  were  found  across  the  board  on  the  ’ 
self-ratings.  Larger  means  reflect  “more”  of  the  characteristic:  comfort  using  computers  (M  pretest 
=  4.4,  M  posttest =  5.3,  M  diff—  -9;  t  (24)  =  2.8,  E  <  .005),  knowledge  about  installing  RAM  (M 
pretest =  2.2,  M  posttcst=  5.3,  M  diff=  3.1;  t  (24)  =  8.8,  e  <  .0001),  confidence  in  apply  that 
knowledge  (M  pretest =  3.0,  M  posttest =  5.0,  M  diff~  2.0;  t  (24)  =  6.3,  e  <~  .0001),  and  confidence  in 
computer  skills  overall  (M  pretest  =  3.9,  M  posttest  =  4.6,  M  diff=  .7;  t  (24)  =  4.0,  p  <  .0005).  There 
was  no  effect  of  humor  on  any  of  the  aforementioned  difference  scores. 

Student  instructional  preferences  changed  after  experiencing  the  courseware  compared  to 
before  the  experience.  More  students  preferred  the  lesson  on  installing  RAM  be  presented  as 
CBT  than  either  group  lecture  or  CAI  after  they  had  the  XADDA  experience.  Nearly  twice  as 
many  students  chose  CBT  as  their  instructional  preference  after  the  lesson  than  before  the  lesson. 
Retention  data  collected  from  21  of  the  participants  revealed  no  significant  loss  in  learning 
(M  retention  -  12.1;  t  (19)  =  .24,  e  =  NS). 

Results  from  the  training  assessment  survey  are  presented  in  Figure  %  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 


22 


Training  Assessment  Survey:  Installing  RAM 


Figure  Training  Assessment  Survey  Resutls  for  Installing  RAM  Courseware 


the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  ratings  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 

Overall  the  courseware  received  high  ratings  of  acceptability  from  the  students.  Students 
rated  all  aspects  of  the  courseware  as  acceptable,  with  the  exception  of  repetition  (M  =  4.7). 

Students  who  experienced  the  humorous  courseware  reported  a  more  positive  learning 
experience,  as  expected.  Use  of  humorous  graphics  affected  students’  perception  of  the 
multimedia  and  motivational  capacity  of  the  courseware.  Students  receiving  the  humorous 
courseware  perceived  the  multimedia  as  contributing  to  learning  (M  Humor  =  6.5,  Mno  Humor  =  5.8; 
t(23)  =  1 .7,  p  <  .05)  and  the  courseware  as  being  more  motivating  (M  Humor =  6.5,  Mn0  Humor  = 
5.6;  t(23)  =  1.8,  p  <  .04)  than  the  students  receiving  the  non-humorous  courseware. 

Discussion 

The  Installing  RAM  courseware  produced  a  125%  increase  in  knowledge  gain.  The 
courseware  was  completed  on  average  in  32  minutes  producing  a  37%  savings  in  class  time. 
Students  showed  a  strong  preference  for  self-paced  CBT  after  their  experience  with  the 
courseware.  Plus,  after  their  experience,  students  self-reported  being  more  comfortable  learning 
from  computers. 

The  only  negative  reaction  students  had  to  the  courseware  was  the  repetitiveness  of 
content.  Although  the  repetitiveness  was  perceived  as  less  than  stimulating,  it  did  enhance 
learning.  For  instance,  as  one  student  commented,  “I  liked  it ! !  Now  I  know  how  to  insert  RAM. 
Drilled  it  into  my  head.”  It  is  likely  that  students  had  somewhat  biased  perceptions  because  they 
were  enrolled  in  a  CIS  class. 
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Hands-on  Practice  and  ICW:  Administering  a  Z-track  Injection  Courseware 

A  nursing  instructor  at  El  Paso  Community  College,  in  El  Paso,  developed  a  lesson 
covering  the  steps  to  administering  a  Z-track  intramuscular  injection.  The  objective  of  the  lesson 
was  to  identify  the  steps  used  to  administer  medication  via  the  Z-track  intramuscular  injection. 
The  lesson  had  two  tiers,  each  with  three  sections. 

The  participating  class  was  randomly  split  into  two-groups  between  completion  of  the 
courseware  and  completion  of  the  post-measures.  One  group  completed  the  evaluation  booklet 
and  then  practiced  administering  a  Z-track  injection  into  a  gel  cushion  (see  Figured).  The  other 
group  practiced  administering  the  injection  then  completed  the  post-measures  of  self-report 
levels  of  knowledge  and  confidence  administering  a  Z-track  injection.  Although  the  sample  size 
was  small,  we  expected  to  see  higher  confidence  ratings  for  the  group  that  had  hands-on  practice 
before  they  completed  the  evaluation  booklet  compared  to  the  group  that  practiced  after  they 
completed  the  booklet. 


Figure  0  -  Nursing  Student  Hands-on  Practice  Administering  a  Z-track  Injection 


Multimedia 

The  instructor  used  graphics  scanned  from  a  nursing  textbook.  Adobe  Premiere®  was 
used  to  crop  and  resize  the  graphics. 

Computer  Equipment  and  Implementation 

A  computer  lab  at  the  Mission  Del  Paso  Campus  was  the  site  for  the  evaluation.  The 
computer  lab  contained  25  Pentium  166  MHz  computers.  The  computers  were  in  a  workstation 
configuration  with  sunken  monitors  and  glare  shields  (See  Figure  $).  A  computer  attached  to  a 
projector  was  used  to  demonstrate  how  to  navigate  the  interfaces. 
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Computer  lab  staff  disabled  the  “write  protect”  program  on  each  computer.  This  allowed 
the  researcher,  an  assistant,  and  the  instructor  to  load  the  XAIDA  Deliver  software,  courseware 
files,  and  set  the  parameters  for  the  monitor  color  pallet  (24-bit)  and  display  area  (640  X  480). 

Participants 

Eleven  students  enrolled  in  a  summer  nursing  course  taught  by  another  instructor 
participated  in  the  evaluation. 

Evaluation  Procedures 

Participants  reported  to  their  regularly  scheduled  classroom.  They  were  then  instructed  to 
report  to  the  computer  lab  located  in  the  library.  The  class  was  given  a  demonstration,  verbal,  and 
written  instructions  on  how  to  open  the  lesson,  sign-in,  and  use  the  interfaces. 

Participants  completed  pre-measures,  the  lesson,  and  remained  in  practice  until  they 
mastered  100%  of  the  facts.  The  posttest  was  administered  immediately  following  the  practice 
exercises.  Half  of  the  participants  completed  the  self-report  knowledge  and  confidence  items  and 
training  assessment  survey  immediately  following  the  practice  exercises.  The  other  half 
completed  the  self-report  items  and  training  assessment  survey  after  they  had  hands-on  practice 
administering  a  Z-track  injection. 


Results 

Table  9  contains  the  mean  results  from  the  classroom  evaluation.  The  courseware 
produced  a  statistically  significant  increase  in  overall  test  scores  (t(10)  =  4.9,  p  <  .0005)  on  the 
13-item  written  test.  The  courseware  also  produced  significant  increases  in  self-reported 
knowledge  of  (M  pretest =  1 .5,  M  posttest  =  5.2,  M  diff=  3.7;  t  (10)  =  26.5,  p  <  .0001),  and  confidence 
in  (M  pretest =  1.6,  M  posttest =  4.9,  M  <jifr=  3.3;  t  (10)  =  7.6,  p  <  .0001)  administering  the  Z-track 
injection.  No  difference  was  found  across  the  evaluation  in  self-reported  confidence  in  overall 
nursing  skills  (M  pretest=  4.8,  M  posttest  =  5.0,  M  diff=  .2;  t  (10)  =  1.4,  p  =  NS).  The  self-report 
items  were  rated  on  7-point  scales  where  “1”  represented  not  al  all,  “3”  represented  somewhat, 
“5”  represented  comfortable,  knowledgeable,  or  confident,  and  “7”  represented  not  at  all. 
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Table  9.  Mean  Results  from  Evaluating  the  Z-track  Injection  Administration  Courseware 


■ 

13-point 

Knowledge 

Pretest 

score 

13-point 

Knowledge 

Posttest 

score 

Knowledge 
Gain  score 

Pre-rating 
of  comfort 
using 
computers 

1-  not  at  all, 
7-  very 

Post-rating 
of  comfort 
using 
computers 

1-  not  at  all, 

7-  very 

Change  in 
comfort 
rating  using 
computers 

1-  not  at  all, 
7-  very 

Instructional 

preference 

Time  on  task 
in  minutes 
(n  =  10) 

6.0 

(sd  =  2.6) 

10.5 

(sd  =  2.3) 

4.5 

(E  < .0005) 

4.7 

(sd=  1.7) 

5.2 

(sd  =  1.1) 

.5 

(E  =  NS) 

Pre-measure 
20%  CBT 

50%  Lecture 
30%  CAI 
Post-measure 
50%  CBT 

30%  Lecture 
20%  CAI 

36.4 

(range  30-45) 

Student  instructional  preferences  changed  dramatically  after  experiencing  the  computer- 
based  lesson  compared  to  before  the  experience.  Twenty  percent  of  students  preferred  the  lesson 
on  Z-track  injection  presented  as  self-paced  CBT  before  actually  experiencing  the  lesson. 
Following  the  lesson,  50%  of  the  students  preferred  the  lesson  as  self-paced  CBT. 

There  was  no  effect  of  hands-on  practice  (before  post-measures  versus  after  post¬ 
measures)  for  knowledge  and  confidence  self-ratings.  However,  average  self-ratings  tended  to  be 
in  the  expected  direction  (Knowledgeable:  M  Hands-on  =  3.8,  M  No  hands-on  =  3.7;  Confident: 

M-  Hands-on  —  3.4,  M  No  hands-on  —  3.2). 
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Figure  J  0  >  Training  Assessment  Survey  Results  for  Z-track  Injection  Courseware 
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Results  from  the  training  assessment  survey  are  presented  in  Figure  Id.  Students  rated  the 
survey  items,  aspects  of  sound  instruction,  on  a  7-point  scale.  Reciprocal  descriptors  anchored 
the  scale  end-points,  for  example,  sufficient  and  insufficient,  adequate  and  inadequate, 
stimulating  and  boring.  Low  numbers  signified  unacceptable  ratings  and  high  numbers  signified 
acceptable  ratings.  Average  ratings  below  “5”  indicate  aspects  of  the  courseware  that  need 
improvement. 

Lesson  relevance  received  a  sufficiently  high  rating  (M  =  6.9)  to  validate  all  ratings. 
Overall  the  courseware  received  very  high  acceptability  ratings  from  the  students.  Students  rated 
all  aspects  of  the  courseware  near  “6.”  Only  the  sufficient  amount  of  interaction  for  learning  (M 
=  5.6)  received  an  average  rating  below  “6.”  None  of  the  items  received  an  unacceptable  rating 
below  “5.” 

Discussion 

Administering  a  Z-track  Injection  courseware  produced  a  75%  increase  in  student  factual 
knowledge  about  the  injection  procedure.  An  average  of  28%  savings  in  class  time  was  found 
with  the  courseware.  The  courseware  also  provided  a  positive  learning  experience  for  the 
students. 

Twice  as  many  students  expressed  a  preference  for  self-paced  CBT  after  they  completed 
the  courseware.  The  courseware  produced  increases  in  students’  knowledge  and  confidence 
administering  a  Z-track  intramuscular  injection.  The  point  in  instruction  at  which  hands-on 
practice  occurred  had  no  effect  on  students’  self-ratings  of  knowledge  and  confidence 
administering  the  injection. 


DISCUSSION 

Students’  Needs 

Students  need  more  computer-based  instruction  for  learning.  This  is  evidenced  in  the 
effectiveness  with  which  the  students  learned  from  the  ICW  developed  by  their  instructors.  All 
XAIDA  courseware  was  instructionally  sound.  Instructional  effectiveness  of  the  courseware  was 
evaluated  as — how  much  the  students  learned,  the  efficiency  with  which  they  learned,  and  how 
they  reacted  to  the  learning  experience.  XAIDA  courseware  produces  a  broad  range  of 
knowledge  gain  scores.  Increases  in  knowledge  gain  scores  from  8%  to  125%  were  found  across 
the  different  topics  presented  in  the  courseware. 

The  eight  evaluations  reveal  a  substantial  timesaving  in  almost  every  classroom.  The  best 
case  was  an  average  timesaving  of  40%  compared  to  a  nominal  50-minute  class  period. 
Timesaving  in  the  classroom  benefits  students.  Extra  time  allows  for  one-on-one  instruction  and 
additional  hands-on  practice.  The  worst  case  was  zero  savings  when  using  the  courseware  as 
CAL  However,  there  are  benefits  to  students,  as  well  as  instructors,  when  ICW  is  implemented  as 
CAI.  For  instance,  students  get  exposure  to  computer-based  instruction  with  CAI  in  the  cases 
where  a  department  may  not  have  computer  resources  to  deliver  courseware  as  stand-alone 
instruction.  In  addition,  instruction  is  standardized.  Courseware  can  be  used  for  remedial 
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purposes  as  self-paced  CBT  by  students  who  need  to  makeup  a  class  or  as  tutorials  for  students 
who  want  to  review  a  class. 

The  117  community  college  students,  faculty,  and  staff  who  participated  in  the  classroom 
evaluations  report  a  generally  positive  learning  experience.  The  students  show  a  readiness  for 
computer-based  instruction  as  part  of  their  regular  curriculum.  Before  participating  in  the 
evaluation,  61.8%  of  students  report  being  “comfortable  using  computers.”  After  their 
participation,  75.3%  of  students  report  being  “comfortable  using  computers.” 

Of  real  interest  is  the  shift  in  instructional  preferences  expressed  by  students.  Seventy- 
three  percent  of  student-participants  express  a  preference  for  computer-based  instruction  over 
traditional  classroom  instruction  after  they  experience  ICW.  It  seems  that  once  students  realize 
that  they  can  learn  from  computers,  then  they  are  more  willing  to  use  computer-based  instruction. 
The  chart  below  shows  the  breakdown  of  student  preferences  into  instructional  categories.  There 
is  a  noticeable  shift  in  preferences  toward  self-paced  CBI.  After  participating  in  the  evaluation 
55%  of  students  show  a  preference  for  self-paced  CBI;  whereas,  before  the  evaluation  only  40% 
of  students  show  a  preference  for  self-paced  CBI. 


Instructors’  Needs 

Community  college  instructors  have  the  wherewithal  to  develop  ICW  for  use  in  their 
classrooms.  However,  the  instructor  alone  can  not  complete  the  development  and  implementation 
process.  The  process  requires  technical  support  and  assistance  from  computer  staff  and  perhaps 
instructional  technology  staff.  Instructors  need  resources  and  training  to  become  proficient  at 
courseware  development  and  knowledgeable  about  courseware  implementation  and  evaluation. 
Instructors  need  to  find  additional  time  to  develop  ICW,  have  access  to  multimedia  resources, 
know  what  it  means  to  audit  computer  lab  equipment,  and  work  with  the  computer  lab  staff  at  the 
start  of  courseware  development  and  throughout  the  implementation  process. 
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What’s  Next? 


Below  is  an  equation  for  a  successful  program  aimed  at  empowering  instructors  with  the 
skills  to  develop  and  implement  computer-based  instruction  in  community  college  curricula. 


Instructor  as  an 
ICW  Developer 


Technical  — — 
Support  mm 


Computer 

Support 


ICW  in  the  classroom 


Continued  increase  in  the  use  of  computer-based  instruction  to  facilitate  learning  at 
vocational  and  technical  institutions  requires  some  planning  on  the  part  of  those  involved  in  the 
early  programs.  Faculty  and  staff  within  the  educational  institutions  need  to  be  included  in 
planning  the  movement  toward  computerized  instruction.  As  well,  organizations  that  hire 
graduates  should  be  included  in  the  planning.  A  potential  exists  to  form  partnerships  with 
industry  for  courseware  development  and  distribution.  Interactive  multimedia  courseware 
developed  by  community  college  instructors  for  classroom  training  is  likely  applicable  to  training 
programs  in  industry. 

Critical  to  successfully  incorporating  computers  in  instruction  is  transforming  “new 
programs”  into  “new  ways  of  thinking.” 
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APPENDIX  A  EXAMPLE  OF  AN  EVALUATION  BOOKLET 
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The  instructional  effectiveness  of  the  computer-based  training  lessons  developed  with  the 
Experimental  Advanced  Instructional  Design  Advisor  (XAIDA)  is  of  interest  to  the 
Alamo  Community  College  District.  Mei  Technology  Corporation  has  been  tasked  with 
evaluating  the  effectiveness  of  XAIDA  as  it  contributes  to  technical  training. 


PROJECT  DESCRIPTION 


You  will  experience  a  computer-based  lesson.  Before  and  after  the  experience  written 
performance  data  will  be  collected.  The  data  you  provide  will  be  used  to  help  improve 
the  instruction  others  receive  at  the  community  college  level. 

The  data  you  provide  is  completely  confidential.  We  ask  that  you  not  place  any  identifying 
marks  on  the  response  sheets.  Information  collected  will  not  be  part  of  your  official  record. 
Only  group  statistics  will  be  reported.  Information  will  not  be  divulged  to  anyone  who  is 
not  a  member  of  the  research  team. 

Your  voluntary  participation  is  sincerely  appreciated.  There  are  no  potential  risks  associated 
with  participation  in  this  research.  You  may  choose  not  to  participate  at  any  time. 


I  agree  to  participate  in  the  research  project  described  above. 


Signature  Date 
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Please  complete  the  items  below 


1 .  Choose  the  way  you  would  prefer  to  learn  how  to  install  RAM  (circle  one): 

(A)  self-paced,  computer-based  training 

(B)  group-paced,  instructor-lead  class  lecture 

(C)  instructor-paced,  computer-assisted  learning 


2.  Rate  your  comfort  learning  from  computers  (circle  the  number  that  represents  your  response). 

1 - 2 - -3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Comfortable  Very 

comfortable  comfortable  comfortable 

3.  Rate  your  knowledge  of  how  to  install  RAM. 

1 - 2 — - - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Knowledgeable  Very 

knowledgeable  knowledgeable  knowledgeable 

4.  Rate  your  confidence  in  applying  your  knowledge  to  install  RAM. 

1 - 2- - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Confident  Very 

confident  confident  confident 

5.  How  confident  are  you  in  your  overall  computer  skills? 

1 - 2 - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Confident  Very 

confident  confident  confident 

6.  Do  not  place  you  name  on  this  booklet.  Instead,  enter  the  last  four  digits  of  you  SS# 


7.  Turn  the  page  and  complete  the  pretest.  Wait  for  instructions  after  completing  the  pretest. 
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Pretest  A 


1 .  List  the  four  steps  to  installing  RAM 


2.  The  “skin”  of  the  CPU  refers  to  the 

A.  RAM  chips 

B.  SIMM  chips 

C.  case 

D.  clear  plastic  cover 


3.  RAM  stands  for. . . . 

A.  Read  And  Memorize 

B.  Random  Access  Memory 

C.  Read  And  Write  Memory 

D.  None  of  the  above 


4.  The  proper  way  to  handle  a  SIMM  chip  is  . . . 

A.  only  in  the  wrapper 

B.  by  the  top  and  bottom 

C.  by  the  pins 

D.  by  the  sides 


5.  _ is  the  standard  RAM  found  in  home  computers  that  have  not  been  upgraded. 


A.  8-16  MB 

B.  200  Hz 

C.  16-32  MB 

D.  100  Hz 
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6.  Where  are  the  RAM  slots  located  on  the  motherboard? 


A.  usually  opposite  the  power  supply 

B.  directly  next  to  the  power  supply 

C.  on  the  microprocessing  chip 

D.  between  expansion  slots 


7.  RAM  speed  is  measured  in. . . 

A.  milliseconds 

B.  nanoseconds 

C.  microseconds 

D.  picoseconds 

8 .  A  WARNING  when  installing  RAM  is . . . 

A.  don’t  touch  the  CPU 

B.  don’t  touch  the  pins  on  the  SIMM  chip 

C.  don’t  unplug  the  computer 

D.  don’t  close  the  software  applications 

9.  SIMM  chips  are  typically. . . . 

A.  32-pin  and  70-pin 

B.  8-pin  and  16-pin 

C.  30-pin  and  72-pin 

D.  16-pin  and  256-pin 

10.  When  installing  and  setting  the  SIMM  chip  you  need  to  pay  attention  to: 

A.  pin-one  position 

B.  proper  handling  of  the  chip 

C.  the  notch 

D.  all  of  the  above 


1 1 .  What  is  the  last  thing  you  do  before  closing  the  CPU? 

A.  reformat  the  hard  drive 

B.  turn  the  computer  on  as  a  memory  test 

C.  discharge  the  static  electricity 

D.  all  of  the  above 
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TO  OPEN  THE  LESSON: 


Type  your  initials  in  the  NAME  box  and  the  last  four  digits  of  your  SS#  in  the  ID  box. 

Click  the  SIGN  IN  button  on  the  left. 

Read  the  following  instructions  while  waiting  for  the  lesson  to  open. 

Click  the  NEXT  button  to  move  forward  through  the  lesson. 

Click  the  BACK  button  to  review  the  previous  screen. 

To  continue  the  lesson  after  viewing  a  graphic,  click  the  CONTINUE  button. 

When  in  practice,  make  your  response  to  an  item  and  click  the  DONE  button. 

To  move  on  to  the  next  practice  item,  click  the  OK  button  (perhaps  more  than  once)  after 
viewing  your  feedback. 

If  you  want  to  return  to  the  lesson  to  look  up  an  answer  click  the  LOOK  UP  button.  You  need  to 
click  the  RETURN  TO  PRACTICE  button  to  make  your  response. 

COMPLETE  THE  LESSON  AND  PRACTICE  EXERCISES 


Write  any  comments  you  have  about  the  lesson  here: 


AFTER  THE  LESSON  TURN  THE  PAGE  AND  TAKE  THE  POSTTEST 
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Posttest  B 


1 .  List  the  four  steps  to  installing  RAM 


2.  The  most  important  pin  on  a  SIMM  chip  is: 

A.  the  last  pin 

B.  Pin-16 

C.  internal  power  pin 

D.  pin-one 


3 .  CPU  stands  for . . . . 

A.  Common  Processing  Unit 

B.  Common  Program  Unit 

C.  Central  Processing  Unit 

D.  None  of  the  above 


4.  You  are  mishandling  the  SIMM  chip  if  you  hold  it . . . 

A.  in  its  anti-static  wrap 

B.  by  the  pins 

C.  by  the  sides 

D.  all  of  the  above 


5.  The  maximum  capacity  of  RAM  can  be  expanded  to... 

A.  256  MB 

B.  200  Hz 

C.  64  MB 

D.  100  Hz 
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6.  RAM  slots  are  located _ 

A.  on  the  power  supply 

B.  directly  next  to  the  power  supply 

C.  on  the  microprocessing  chip 

D.  on  the  motherboard 


7.  Nanoseconds  are  used  to  measure: 

A.  SIMM  chip  size 

B.  SIMM  chip  capacity 

C.  SIMM  chip  speed 

D.  all  of  the  above 


8 .  AN OTICE  when  installing  RAM  is . . . 

A.  don’t  forget  to  discharge  the  static  electricity 

B.  don’t  forget  to  check  the  manual 

C.  don’t  unplug  the  computer  from  the  wall  socket 

D.  don’t  close  the  software  applications 

9.  SIMM  chips  are  typically. . . . 

A.  32-pin  and  70-pin 

B.  8-pin  and  16-pin 

C.  30-pin  and  72-pin 

D.  1 6-pin  and  256-pin 

1 0.  Pin-one  position: 

A.  defines  size  and  speed  of  the  SIMM  chip 

B.  determines  proper  handling  of  the  chip 

C.  determines  proper  SIMM  chip  alignment 

D.  defines  the  type  of  SIMM  chip 

1 1 .  What  is  the  first  thing  you  do  before  opening  the  CPU? 

A.  unplug  all  external  peripherals 

B.  turn  the  computer  off 

C.  open  SIMM  installation  software 

D.  discharge  static  electricity 
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Circle  the  number  that  represents  your  response. 

1 .  Rate  your  comfort  learning  from  computers. 

1 - 2 - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Comfortable  Very 

comfortable  comfortable  comfortable 


2.  Rate  your  knowledge  of  how  to  install  RAM. 

1 - 2- - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Knowledgeable  Very 

knowledgeable  knowledgeable  knowledgeable 


3.  Rate  your  confidence  in  applying  your  knowledge  to  install  RAM. 

1 - 2 - 3 - 4 - 5 - 6 - 7 

Not  at  all  Somewhat  Confident  Very 

confident  confident  confident 


4.  How  confident  are  you  in  your  overall  computer  skills? 

-7 
Very 
confident 


1 - 

- 2- — 

- 3 - 

— 4 - 

- 5 - 

Not  at  all 

Somewhat 

Confident 

confident 

confident 

CONTINUE 
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Circle  the  NUMBER  that  best  represents  your  opinion.  Note  the  scale  end-points! 

1 .  The  lesson  features  (buttons,  hot  boxes,  practice)  were _ to  use. 

Easy  1 - 2 - 3 - 4 - 5 - 6 - 7  Difficult 

2.  The  lesson  objective  was  _ presented. 

Clearly  1 - 2 - 3 - 4 - 5 - 6 - 7  Not  clearly 

3.  The  instructional  sequence  was _ in  keeping  my  attention. 

Inadequate  1 - 2 - 3 - 4 - 5 - 6 - 7  Adequate 

4.  The  lesson  was  _ to  understand  than  I  would  have  liked  it  to  be. 

More  difficult  1 - 2 - 3 - 4 - 5 - 6 - 7  Easier 

5 .  Repetition  of  lesson  content  was. . . 

Stimulating  1 - 2 - 3 - 4 - 5 - 6 - 7  Boring 

6.  Terms,  concepts,  and  information  that  were  important  to  know  were  emphasized... 

Effectively  1 - 2 - 3 - 4 - 5 - 6 - 7  Ineffectively 

7.  Question-and-answer  sessions  were _ for  learning. 

Inadequate  1 - 2 - 3 - 4 - 5 - 6 - 7  Adequate 

8.  The  multimedia  (graphics,  audio,  video) _  learning. 

Add  to  1 - 2 - 3 - -4 - 5 - 6 - 7  Detracted  from 

9.  The  amount  of  interaction  (with  students,  instructor,  computer)  was  _ for  learning. 

Insufficient  1 - 2 - 3 - 4 - 5 - 6 - 7  Sufficient 

1 0.  The  pace  of  the  lesson  was _ for  learning. 

Inadequate  1 - 2 - 3 - 4 - 5 - 6 - 7  Adequate 

1 1 .  Overall  the  lesson  was... 

Motivating  1 - 2 - 3 - 4 - 5 - 6 - 7  Unmotivating 

12.  This  lesson  was _ to  my  education. 

Irrelevant  1 - 2 - -3 - 4 - 5 - 6 - 7  Relevant 

13.  I  would  prefer  that  the  lesson  on  how  to  install  RAM  be  presented  as  (circle  one): 

(A)  self-paced,  computer-based  training 

(B)  group-paced,  instructor-lead  class  lecture 

(C)  instructor-paced,  computer-assisted  learning 
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